Skip to content

[UR] Implement IPC physical_mem API in UR#22115

Open
ldorau wants to merge 7 commits into
intel:syclfrom
ldorau:UR_Implement_IPC_physical_mem_API_in_UR
Open

[UR] Implement IPC physical_mem API in UR#22115
ldorau wants to merge 7 commits into
intel:syclfrom
ldorau:UR_Implement_IPC_physical_mem_API_in_UR

Conversation

@ldorau
Copy link
Copy Markdown
Contributor

@ldorau ldorau commented May 26, 2026

Implement urIPCGetPhysMemHandleExp, urIPCPutPhysMemHandleExp,
urIPCOpenPhysMemHandleExp, and urIPCClosePhysMemHandleExp across all adapters.

Level Zero v2 adapter (full implementation, Linux only)

Uses DMA-BUF fds exported via zePhysicalMemGetProperties and cross-process
fd duplication via pidfd_getfd(2) (Linux 5.6+). New files v2/physical_mem.hpp
and v2/physical_mem.cpp replace the shared physical_mem.cpp for the v2 build.
UR_DEVICE_INFO_IPC_PHYSICAL_MEMORY_SUPPORT_EXP is true on Linux, false on Windows.

Level Zero v1 adapter — unsupported stubs; device info returns false.

CUDA, HIP, OpenCL, Native CPU, Offload adapters — unsupported stubs; device info
returns false where applicable.

Conformance tests — four new test files (urIPCGetPhysMemHandleExp,
urIPCPutPhysMemHandleExp, urIPCOpenPhysMemHandleExp, urIPCClosePhysMemHandleExp)
with shared fixtures; urPhysicalMemCreate updated to cover the ENABLE_IPC flag.

@ldorau ldorau requested a review from slawekptak May 26, 2026 09:53
Comment thread unified-runtime/include/unified-runtime/ur_api.h
Comment thread unified-runtime/source/adapters/level_zero/memory.cpp Outdated
Comment thread unified-runtime/source/adapters/opencl/common.hpp Outdated
Comment thread unified-runtime/test/conformance/virtual_memory/ze_ipc_phys_mem_e2e.cpp Outdated
@ldorau ldorau force-pushed the UR_Implement_IPC_physical_mem_API_in_UR branch from 70701a0 to 523e104 Compare May 26, 2026 12:02
@ldorau
Copy link
Copy Markdown
Contributor Author

ldorau commented May 26, 2026

Rebased

@ldorau ldorau requested a review from slawekptak May 26, 2026 12:02
@ldorau ldorau force-pushed the UR_Implement_IPC_physical_mem_API_in_UR branch from 523e104 to 4cbcfa8 Compare May 26, 2026 14:30
@ldorau ldorau marked this pull request as ready for review May 27, 2026 08:22
@ldorau ldorau requested review from a team as code owners May 27, 2026 08:22
@ldorau ldorau requested a review from pbalcer May 27, 2026 08:22
@ldorau
Copy link
Copy Markdown
Contributor Author

ldorau commented May 27, 2026

The Offload CI job fails because of #22125

@ldorau ldorau force-pushed the UR_Implement_IPC_physical_mem_API_in_UR branch from a9eb70a to 6dac91d Compare May 27, 2026 11:12
@ldorau
Copy link
Copy Markdown
Contributor Author

ldorau commented May 27, 2026

Rebased

@ldorau ldorau force-pushed the UR_Implement_IPC_physical_mem_API_in_UR branch from 29aee2c to 4a61124 Compare May 27, 2026 13:01
@ldorau ldorau marked this pull request as draft May 27, 2026 20:28
@ldorau ldorau force-pushed the UR_Implement_IPC_physical_mem_API_in_UR branch from 6113038 to 572be16 Compare May 28, 2026 08:49
Comment thread unified-runtime/source/adapters/level_zero/physical_mem.cpp Outdated
Comment thread unified-runtime/source/adapters/level_zero/physical_mem.cpp Outdated
@ldorau ldorau force-pushed the UR_Implement_IPC_physical_mem_API_in_UR branch 5 times, most recently from 75afc3f to 3c08b2b Compare May 29, 2026 13:35
@ldorau ldorau requested a review from nrspruit May 29, 2026 14:45
@ldorau
Copy link
Copy Markdown
Contributor Author

ldorau commented May 29, 2026

The Offload CI job fails because of #22125

@ldorau ldorau marked this pull request as ready for review May 29, 2026 14:55
@ldorau ldorau requested a review from kswiecicki May 29, 2026 15:03
@ldorau ldorau force-pushed the UR_Implement_IPC_physical_mem_API_in_UR branch from 879915f to dc8e61e Compare June 1, 2026 14:08
@ldorau ldorau force-pushed the UR_Implement_IPC_physical_mem_API_in_UR branch from dc8e61e to 95c1d4a Compare June 1, 2026 14:13
@ldorau
Copy link
Copy Markdown
Contributor Author

ldorau commented Jun 1, 2026

@nrspruit please review 2 files: unified-runtime/source/adapters/level_zero/v2/physical_mem.cpp and unified-runtime/source/adapters/level_zero/v2/physical_mem.hpp

@ldorau
Copy link
Copy Markdown
Contributor Author

ldorau commented Jun 1, 2026

The Offload CI job fails because of #22125

@ldorau ldorau force-pushed the UR_Implement_IPC_physical_mem_API_in_UR branch 2 times, most recently from 88e952f to fe60cba Compare June 1, 2026 19:17
Comment thread unified-runtime/source/adapters/level_zero/v2/physical_mem.cpp Outdated
Comment thread unified-runtime/source/adapters/level_zero/v2/physical_mem.cpp Outdated
ldorau added 2 commits June 3, 2026 08:58
Add four new function specs to exp-inter-process-communication.yml
and update registry.yml to support IPC operations on physical memory
objects:
- urIPCGetPhysMemHandleExp: export an IPC handle for a physical_mem
- urIPCPutPhysMemHandleExp: release the exported IPC handle
- urIPCOpenPhysMemHandleExp: import an IPC handle into a physical_mem
- urIPCClosePhysMemHandleExp: close the imported physical_mem

Also add ENABLE_IPC flag to ur_physical_mem_flags_t in virtual_memory.yml
to allow allocating physical memory that can be shared via IPC handles.

Signed-off-by: Lukasz Dorau <lukasz.dorau@intel.com>
Add auto-generated files based on the IPC physical_mem spec changes
(urIPCGetPhysMemHandleExp, urIPCPutPhysMemHandleExp,
urIPCOpenPhysMemHandleExp, urIPCClosePhysMemHandleExp,
UR_PHYSICAL_MEM_FLAG_ENABLE_IPC, UR_DEVICE_INFO_IPC_PHYSICAL_MEMORY_SUPPORT_EXP):

- ur_api.h, ur_api_funcs.def, ur_ddi.h: function declarations
- ur_print.h/hpp, ur_print.cpp: printing support
- ur_api.cpp: API dispatch stubs
- ur_ldrddi.cpp, ur_libapi.cpp: loader implementations
- ur_trcddi.cpp, ur_valddi.cpp: tracing and validation layers
- ur_mockddi.cpp: mock adapter
- loader.def.in, loader.map.in: loader symbol exports
- level_zero ur_interface_loader.cpp/.hpp: Level-Zero DDI table entries
- urinfo.hpp: updated urinfo tool

Signed-off-by: Lukasz Dorau <lukasz.dorau@intel.com>
@ldorau ldorau force-pushed the UR_Implement_IPC_physical_mem_API_in_UR branch from 3acc55a to a559824 Compare June 3, 2026 09:57
@ldorau ldorau requested review from kswiecicki and slawekptak June 3, 2026 10:00
Implement urIPCGetPhysMemHandleExp, urIPCPutPhysMemHandleExp,
urIPCOpenPhysMemHandleExp, and urIPCClosePhysMemHandleExp across all
adapters.

Level Zero v2 adapter (full implementation, Linux only):
- physical_mem.hpp: extend ur_physical_mem_handle_t_ with Device, Size,
  EnableIpc fields
- v2/physical_mem.hpp: add ZeIPCPhysMemHandleData struct (pid + DMA-BUF
  fd + size) for cross-process handle exchange via pidfd_getfd(2)
- v2/physical_mem.cpp: chain ze_external_memory_export_desc_t in
  urPhysicalMemCreate when ENABLE_IPC flag is set; extend
  urPhysicalMemGetInfo to return context, device, size and properties;
  implement the four IPC functions using ZeIPCPhysMemHandleData
- device.cpp: report UR_DEVICE_INFO_IPC_PHYSICAL_MEMORY_SUPPORT_EXP
  as true on Linux, false on Windows

Level Zero v1 adapter (unsupported stubs):
- physical_mem.cpp: return UR_RESULT_ERROR_UNSUPPORTED_FEATURE from all
  four IPC functions; urPhysicalMemCreate always passes EnableIpc=false
- device.cpp: report UR_DEVICE_INFO_IPC_PHYSICAL_MEMORY_SUPPORT_EXP
  as false

CUDA, HIP, OpenCL, Native CPU adapters:
- Return UR_RESULT_ERROR_UNSUPPORTED_FEATURE from all four functions
- Register function pointers in ur_interface_loader.cpp
- Report UR_DEVICE_INFO_IPC_PHYSICAL_MEMORY_SUPPORT_EXP as false

Offload adapter:
- Return UR_RESULT_ERROR_UNSUPPORTED_FEATURE from all four functions
- Register function pointers in ur_interface_loader.cpp

Conformance tests:
- Add urIPCGetPhysMemHandleExp.cpp, urIPCPutPhysMemHandleExp.cpp,
  urIPCOpenPhysMemHandleExp.cpp, urIPCClosePhysMemHandleExp.cpp
  with shared fixtures in urIPCPhysMemHandleExpFixtures.hpp
- Update urPhysicalMemCreate.cpp to cover the ENABLE_IPC flag

Signed-off-by: Lukasz Dorau <lukasz.dorau@intel.com>
Comment thread unified-runtime/source/adapters/level_zero/v2/physical_mem.cpp Outdated
@ldorau ldorau requested a review from slawekptak June 3, 2026 11:39
Use std::optional<ze_external_memory_export_desc_t> so the export
descriptor is only created when UR_PHYSICAL_MEM_FLAG_ENABLE_IPC is
set in urPhysicalMemCreate. The optional lives at function scope,
keeping the pointer valid through the zePhysicalMemCreate call.

Signed-off-by: Lukasz Dorau <lukasz.dorau@intel.com>
@ldorau
Copy link
Copy Markdown
Contributor Author

ldorau commented Jun 3, 2026

Updated with upstream/sycl.

@ldorau
Copy link
Copy Markdown
Contributor Author

ldorau commented Jun 3, 2026

@nrspruit please review

Copy link
Copy Markdown
Contributor

@kswiecicki kswiecicki left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be nice if the urIPCOpen... were to be tested in multi-process testcase. I'm not sure if its possible to integrate it in the current conformance testing framework. Perhaps we would need a SYCL E2E test.

Comment thread unified-runtime/source/adapters/level_zero/v2/physical_mem.cpp Outdated
ldorau added 2 commits June 3, 2026 13:37
zePhysicalMemGetProperties requires the same context that was used
to create the physical memory object.  Use hPhysMem->Context instead
of the caller-supplied hContext, which may be a different context.
Mark hContext as [[maybe_unused]] since it is now unused on Linux.

Signed-off-by: Lukasz Dorau <lukasz.dorau@intel.com>
Replace [[maybe_unused]] ur_context_handle_t hContext with an
unnamed parameter (ur_context_handle_t) in urIPCGetPhysMemHandleExp,
urIPCPutPhysMemHandleExp, and urIPCClosePhysMemHandleExp.

Signed-off-by: Lukasz Dorau <lukasz.dorau@intel.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants